home
***
CD-ROM
|
disk
|
FTP
|
other
***
search
/
Tech Arsenal 1
/
Tech Arsenal (Arsenal Computer).ISO
/
tek-04
/
aie9003.zip
/
NET.DOC
< prev
next >
Wrap
Text File
|
1989-12-29
|
10KB
|
273 lines
BAYSEAN OR OF ANDS NETWORK
(C) Copyright 1989 by Instant Recall
All rights reserved.
Rodger Knaus
Instant Recall
P.O. Box 30134
Bethesda, Md. 20814
(301) 530-0898
PURPOSE
This is a neural network which is designed to find and tune expert
system rules. It can be used when
* The user knows what kinds of facts are relevant to a decision,
but not how the facts combine with each other in producing
particular decision results.
* The user knows some of the rules that combine facts, and wants
to test and refine these, and also discover other rules.
* The user has guessed at confidence factors in rules and wants
to tune them.
OVERVIEW OF NETWORK
The network accepts inputs which are logical propositions and their
negatives, with probability truth values. The logical propositions
do not at this time contain variables.
The 2nd. layer of the network contains AND nodes of input layer nodes.
Each 2nd. layer node is an AND of a different combination of 1st. layer
nodes. The edge to an AND node is 0 if the input to the edge is not
in the AND at the output. The edge is 1 if the input to the edge
is in the AND. The edge has an intermediate value if the input has
some effect on the AND, but not enough to make it completely false
if the input is false. The weights for AND nodes are computed using
the Baysean formula, taking the product of input activations.
Each input is raised to the power of the edge weight from it to the
output and node. This produces a weighted product similar to the
usual weighted sum. The assumption of statistical independence is
not strictly satisfied, but is often approximated, because the
knowledge engineer finds independent propositions solve more
different problems than highly correlated ones.
The 3rd and final layer is a layer of ORs of the AND nodes. The
weights are similar to those to the AND nodes. However, the
activation of an OR is computed using the Bayean formulas for
AND and NOT, plus DeMorgan's Law.
The delta rule is applied to the OR weights. The AND weights
are not changed. This follows the approach of Pao's functional-link
nets. The ANDs form a basis of all the possible logical functions.
They are in effect an enriched set of inputs for a simple 1-layer
trainable network.
Any propositional function of the inputs can be represented as
an OR of ANDs of inputs and their negatives. Therefore there
exists a network that solves any problem of computing the
logical formula that has a given behavior.
The problem of finding this formula can also be done algebraically.
However, the algebra becomes unwieldy with large numbers of inputs.
Furthermore, bad data will produce a false formula in the algebraic
approach, but will generally just introduce a small error into the
network coefficients.
USING THE NETWORK TO IMPROVE EXPERT SYSTEM RULES
Expert system rules can be readily encoded into this network.
The steps are:
1. Write the rules in disjuctive normal form.
2. Create an input node for each proposition in a rule body.
3. Create an output node for each proposition in a rule conclusion.
4. Set the weights according to the definitions above.
Then run data with known outputs through the network. This trains
the weights.
To get expert system rules out of the network, do the following:
1. Set edge weights near 0 to 0 and weights near 1 to 1.
2. Use the weight definitions above to convert each AND and OR
node to a propositional function.
USING THE NETWORK FOR PARAMETRIC PROBLEMS
Statements containing variables can be broken up into a series of
propositions without variables. For example, we can change
computer_budget( X )
into
Computer budget is below $1000.
Computer budget is between $1000 and $2000.
Computer budget is between $2000 and $3000.
Computer budget is between $3000 and $4000.
Computer budget is above $4000.
This process is like what happens in the ear, where a neuron is fired
when and only when the ear receives sound of a particular frequency,
and there is a separate neuron for each frequency.
INSTALLING THE PROGRAM
1. Copy the program to the subdirectory where you want to run it.
2. Uncomprressing the archive file : If you get the program as a
single file BNET.EXE, then type
>bnet
to DOS.
RUNNING THE PROGRAM
You should have the following files in your current subdirectory:
NET.EXE : network executable program
NET.IDB : network prolog internal database
NET.CFG : network configuration file
The other files listed below are for study purposes, but not needed
for running the program:
NET.DOC : network documentation
EXEC.ARI : network source code
Here is a sample run of BNET, with comments :
----------------------------------------------
% Run of BNET
>net <return> % DOS command to run BNET
name of net >> xor.ari % name of file containing network definition
number of cycles >> 5 % number of input sets to train net on
Errors at time 0 : 9 -1.0 % net reports its performance
Errors at time 1 : 9 -1.0
Errors at time 2 : 9 0.0
Errors at time 3 : 9 0.0
Errors at time 4 : 9 -0.6
save file >> save1.ari % file to save final network configuration in
----------------------------------------------
Note the following:
* the "name of net" should be a file containing a network description,
as defined in this document.
* the "save file" should be the file where you want to save the network
in the state after its run. You can read it back in on another run.
* We recommend using the "ari" extension for network definitions.
* We have provided a sample network definition in xor.ari, for the
exclusive or.
LOG FILE
Everything that appears on your screen during a run of net is also
written to a log file called log.log.
TRACING NET BEHAVIOR
The file net.cfg contains, among other things, a set of trace
switches that let you see the net behavior in more detail.
Here are the list of trace switches:
% and_learn_trace. % learning and nodes
% and_trace . % and activations
cycle_trace . % results after each cycle
% net_trace . % general
% not_trace . % NOT nodes
% or_trace . % or activations
% or_learn_trace. % learning or nodes
When a % appears before the flag on in the left col., that trace
is turned off. When there is no % in col. 1, as for cycle_trace,
that trace is on.
CONTROLING NET BEHAVIOR
There are also a couple data items in NET.CFG that control the
net behavior. They are shown below:
learning_rate( 1.0 ). % learning rate
backprop_to_and_edges( off ). % switch -- on or off - on whether to change
% and edges
learning_rate controls the speed of learning by the net. It is a number
between 0 and 1. When 0, no learning occurs. When 1, learning occurs
according to the Baysean OR formula form of the delta rule. See the
file NET.COL for more details on the learning formula.
backprop_to_and_edges is a switch with values off and on. It controls
whether errors are propogated back to the "and" level of the net.
If so, the generalized delta rule is used to compute them, i.e.
the credit for an error assigned to an interior neuron is the
sum of errors made by neurons to which its activation is sent, weighted
by the edge weight to each such neuron. We recommend keeping this off,
making the net behave like Pao's functional linked nets.
TROUBLESHOOTING
If NET does not run on your machine, here are some things to check:
1. AVAILABLE MEMORY: This program is written in Prolog, a relative
memory hog. I do not know exactly how much memory it needs, but
it definitly runs when there are over 500K memory free. If you
have less, because you are running with RAM-resident programs,
a lot of device drivers, etc. remove these memory-consumers
and reboot to provide more free memory.
2. DOS 4.0 OR ABOVE: If you are running with these versions of DOS,
you must include ANSII.SYS/K -- the "K" option on ANSII.SYS --
in your CONFIG.SYS fiile. This resolves a conflict between the
Prolog and DOS keyboard handlers. Put this in your config.sys
file and reboot.
NETWORK DESCRIPTIONS
A network description file should contain facts of the form:
input( DATA_INSTANCE_KEY , INPUT_NEURON_NUMBER, VALUE_OF_INPUT ).
input( DATA_INSTANCE_KEY , OUTPUT_NEURON_NUMBER, DESIRED_VALUE_OF_OUTPUT ).
neuron( KIND , NEURON_NUMBER, DESCRIPTION ).
state( edge, KIND, INPUT_NEURON_NUMBER , OUTPUT_NEURON_NUMBER,
0, INITIAL_WEIGHT).
where KIND is one of : input, and, or, not( INPUT_NEURON_NUMBER )
DESCRIPTION OF NETWORK:
INPUTS : in [0 ,1 ]
INPUT LAYER : inputs and their negatives, i.e. ( 1 - input) 's
MIDDLE LAYER : various ands of inputs
activation( A AND B) = activation( A ) * activation( B)
OUTPUT LAYER : ors of middle layer nodes
activation( A OR B) = 1 - ( 1 - activation( A )) *
( 1 - activation( B) )
INITIALIZATION :
edges to and layer are initialized by:
wji = 0 if input or negative of input i is not in
the and represented by node j
wji = 1 if the input or negative of input i is in
the and represented by node j
edges to or layer are initialized as all 1s
LEARNING :
edges to or layer are updated by delta rule.
edges to and layer are not changed.
- eof -